REMARKS 

This amendment is responsive to the Office Action dated December 28, 2004. 
Claims 1-68 were pending in the application. A conversation on December 20, 2004 
between the examiner and patentee's representative John V. Biemacki resulted in claims 
1-38 and 58-69 being withdrawn due to a provisional election to prosecute claims 39-57. 

Claim 39 is amended herein to remove the term "simulated" for proper antecedent 
basis, and accordingly such an amendment is not a narrowing amendment. 

Claims 70-72 have been added herein. 

Claim 41 is objected to and claims 39-57 stand rejected by the Examiner. 
Applicant traverses the objection of claim 41 and rejections of claims 39-57. 

Claim Objection 

Claim 41 stands objected to under 37 CFR 1.75(c) as being of improper 
dependent form for failing to further limit the subject matter of a previous claim. 
Applicant disagrees with the objection. Claim 41 depends from method claim 40 which 
itself depends from independent claim 39. Claim 41 further limits the subject matter of 
independent claim 39. For example, claim 39 recites that a second activation function is 
selected to form a second layer for a neural network. Dependent claim 41 further limits 
this subject matter of claim 39 by reciting "wherein a second candidate activation 
function type is selected as the second activation function for use used within a second 
layer of the artificial neural network, wherein the selected first candidate activation 
function type is a different function type than the selected second candidate activation 
function type." Claim 41 has been amended to further show the limiting of this subject 
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matter of claim 39, and accordingly the objection with respect to claim 41 should be 
removed and claim 41 should proceed to issuance. 

Claim Rejections - 35 USC § 102 

Claims 39-57 stand rejected under 35 USC § 102(b) as being anticipated by 
Mehrotra et al. (MIT Press 0-262-13328-8, hereinafter referred to as "Mehrotra"). This 
rejection is traversed. 

Claim 39 recites in combination with its other limitations that an artificial neural 
network is built from "a set of different types of candidate activation functions" and that 
a first activation function is selected from the different types of candidate activation 
functions for use within a first layer of the artificial neural network. A second activation 
function is selected from the different types of candidate activation functions for use 
within a second layer of the artificial neural network. As an example of the different 
types of activation function types, the set of candidate activation functions could include 
a SQUARE activation function; TANH activation function; ARCTAN activation 
function; LOGISTIC activation function; GAUSS activation function; and 
EXPONENTIAL activation function. Such activation functions are different types of 
functions as illustrated in the following table: 



Activation Functions 


Activation Function 
Type 


Formula 


SQUARE 


(aH-b*x)*x 


TANH 


a*tanh(b*x) 


ARCTAN 


a*atan(b*x) 
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LOGISTIC 


exp(a*x) / (H-exp(b*x) 


GAUSS 


a*exp(-(b*x)'^) 


EXPONENTIAL 


a*exp(b*x) 



With these many multiple types of activation functions, the first layer could be 
constructed using one type of activation function (e.g., a SQUARE activation function) 
and the second layer could be constructed using a different type of activation function 
(e.g., ARCTAN). 

In contrast, Mehrotra does not disclose the limitations of claim 1 . For example, 
Mehrotra does not disclose building a neural network from a set of different types of 
candidate activation functions as required by claim 1 in combination with its other 
limitations. Instead Mehrotra discloses in his text book only general neural network 
descriptions, such as the back propagation algorithm disclosed on page 76 of Mehrotra. 
This algorithm of Mehrotra contains a neural network whose layers are already 
established. The algorithm of Mehrotra does not contain a set of different types of 
candidate activation functions, and in fact expressly states that in its neural network "each 
node has the same activation function'' (see caption of Figure 3.5 on page 76; emphasis 
added). 

Still further, Mehrotra does not contain a determination of which activation 
function should be selected from a pool of candidate activation functions that differ in 
function type, let alone such a selection determination for different layers (i.e., the first 
and second layers) as required by claim 1. Rather, the model used in Figure 3.5 of 
Mehrotra already includes all input, hidden, and output nodes together with prespecified 
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activation functions before the optimization even is started and therefore does not permit 
the intelligent selection of activation functions during model building. 

Because the neural network of Mehrotra is not built from an intelligent selection 
from a pool of different types of activation functions as required by claim 1 in 
combination with its other limitations^ Mehrotra does not render claim 1 unpatentable. 
Accordingly, claim 1 and its dependent claims are allowable and should proceed to 
issuance. 

Applicant disagrees with other positions in the office action. For example, neural 
network weights do not determine whether the type of a first activation function differs 
from the type of another activation function — that is, unique optimal weights will not 
change the type of activation function with which they are associated. Rather different 
weights may be used to alter the inputs/outputs that are provided to/generated from a 
particular type of activation function. This distinction between activation function type 
and weights is recited in claim 40: ("wherein the first and second selected activation 
functions differ with respect to function type from each other; wherein the first and 
second selected activation functions further differ with respect to weights associated with 
their inputs and outputs."). 

Claim 41 further recites this distinction: "wherein the selected first candidate 
activation function type is a different function type than the selected second candidate 
activation function type because of the function types respectively associated with the 
selected first and second candidate activation functions and not because of weights 
associated with the selected first and second candidate activation functions." 
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Still further, newly added claim 70 recites "wherein nodes in the first layer have a 
different activation function type than nodes in the second layer because different types 
of activation functions were selected from the set of candidate activation functions." This 
is in stark contrast with the algorithm of Mehrotra which expressly states that in its neural 
network "each node has the same activation function^' (see caption of Figure 3.5 on page 
76; emphasis added). 

Because of such distinctions over Mehrotra, claims 40, 41 and 70 are also 
allowable and should proceed to issuance. 

As another example, claim 71 recites that a neural network is constructed in a 
stepwise manner: "wherein the neural network is constructed in a layer-by-layer stepwise 
fashion such that the first layer of the neural network is completed before the second 
layer of the neural network is determined; wherein the type of activation function for the 
second layer is unknown until after the first layer has been completed." In contrast, 
Mehrotra discloses a model (e.g., as shown in Figure 3.5 of Mehrotra) that includes all 
input, hidden, and output nodes together with prespecified activation functions before the 
optimization even is started and therefore does not permit the intelligent selection of 
activation functions on a layer-by-layer basis during a stepwise model building process. 
Because of such distinctions, claim 71 is allowable over Mehrotra and should proceed to 
issuance. 

[Continued on next page] 



13 



CONCLUSION 

For the foregoing reasons, applicant respectfully submits that the pending claims 
should be allowed. Therefore, the examiner is respectfully requested to pass this case to 
issue. 

Respectfully submitted, 
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